CAT-DTI: cross-attention and Transformer network with domain adaptation for drug-target interaction prediction.

Accurate and efficient prediction of drug-target interaction (DTI) is critical to advance drug development and reduce the cost of drug discovery. Recently, the employment of deep learning methods has enhanced DTI prediction precision and efficacy, but it still encounters several challenges. The first challenge lies in the efficient learning of drug and protein feature representations alongside their interaction features to enhance DTI prediction. Another important challenge is to improve the generalization capability of the DTI model within real-world scenarios. To address these challenges, we propose CAT-DTI, a model based on cross-attention and Transformer, possessing domain adaptation capability. CAT-DTI effectively captures the drug-target interactions while adapting to out-of-distribution data. Specifically, we use a convolution neural network combined with a Transformer to encode the distance relationship between amino acids within protein sequences and employ a cross-attention module to capture the drug-target interaction features. Generalization to new DTI prediction scenarios is achieved by leveraging a conditional domain adversarial network, aligning DTI representations under diverse distributions. Experimental results within in-domain and cross-domain scenarios demonstrate that CAT-DTI model overall improves DTI prediction performance compared with previous methods.

3D Multimodal Fusion Network with Disease-induced Joint Learning for Early Alzheimer's Disease Diagnosis.

Multimodal neuroimaging provides complementary information critical for accurate early diagnosis of Alzheimer's disease (AD). However, the inherent variability between multimodal neuroimages hinders the effective fusion of multimodal features. Moreover, achieving reliable and interpretable diagnoses in the field of multimodal fusion remains challenging. To address them, we propose a novel multimodal diagnosis network based on multi-fusion and disease-induced learning (MDL-Net) to enhance early AD diagnosis by efficiently fusing multimodal data. Specifically, MDL-Net proposes a multi-fusion joint learning (MJL) module, which effectively fuses multimodal features and enhances the feature representation from global, local, and latent learning perspectives. MJL consists of three modules, global-aware learning (GAL), local-aware learning (LAL), and outer latent-space learning (LSL) modules. GAL via a self-adaptive Transformer (SAT) learns the global relationships among the modalities. LAL constructs local-aware convolution to learn the local associations. LSL module introduces latent information through outer product operation to further enhance feature representation. MDL-Net integrates the disease-induced region-aware learning (DRL) module via gradient weight to enhance interpretability, which iteratively learns weight matrices to identify AD-related brain regions. We conduct the extensive experiments on public datasets and the results confirm the superiority of our proposed method. Our code will be available at: https://github.com/qzf0320/MDL-Net.

Automated detection of nine infantile fundus diseases and conditions in retinal images using a deep learning system.

Purpose: We developed an Infant Retinal Intelligent Diagnosis System (IRIDS), an automated system to aid early diagnosis and monitoring of infantile fundus diseases and health conditions to satisfy urgent needs of ophthalmologists. Methods: We developed IRIDS by combining convolutional neural networks and transformer structures, using a dataset of 7697 retinal images (1089 infants) from four hospitals. It identifies nine fundus diseases and conditions, namely, retinopathy of prematurity (ROP) (mild ROP, moderate ROP, and severe ROP), retinoblastoma (RB), retinitis pigmentosa (RP), Coats disease, coloboma of the choroid, congenital retinal fold (CRF), and normal. IRIDS also includes depth attention modules, ResNet-18 (Res-18), and Multi-Axis Vision Transformer (MaxViT). Performance was compared to that of ophthalmologists using 450 retinal images. The IRIDS employed a five-fold cross-validation approach to generate the classification results. Results: Several baseline models achieved the following metrics: accuracy, precision, recall, F1-score (F1), kappa, and area under the receiver operating characteristic curve (AUC) with best values of 94.62% (95% CI, 94.34%-94.90%), 94.07% (95% CI, 93.32%-94.82%), 90.56% (95% CI, 88.64%-92.48%), 92.34% (95% CI, 91.87%-92.81%), 91.15% (95% CI, 90.37%-91.93%), and 99.08% (95% CI, 99.07%-99.09%), respectively. In comparison, IRIDS showed promising results compared to ophthalmologists, demonstrating an average accuracy, precision, recall, F1, kappa, and AUC of 96.45% (95% CI, 96.37%-96.53%), 95.86% (95% CI, 94.56%-97.16%), 94.37% (95% CI, 93.95%-94.79%), 95.03% (95% CI, 94.45%-95.61%), 94.43% (95% CI, 93.96%-94.90%), and 99.51% (95% CI, 99.51%-99.51%), respectively, in multi-label classification on the test dataset, utilizing the Res-18 and MaxViT models. These results suggest that, particularly in terms of AUC, IRIDS achieved performance that warrants further investigation for the detection of retinal abnormalities. Conclusions: IRIDS identifies nine infantile fundus diseases and conditions accurately. It may aid non-ophthalmologist personnel in underserved areas in infantile fundus disease screening. Thus, preventing severe complications. The IRIDS serves as an example of artificial intelligence integration into ophthalmology to achieve better outcomes in predictive, preventive, and personalized medicine (PPPM / 3PM) in the treatment of infantile fundus diseases. Supplementary Information: The online version contains supplementary material available at 10.1007/s13167-024-00350-y.

Estimating Addiction-Related Brain Connectivity by Prior-Embedding Graph Generative Adversarial Networks.

The study of nicotine addiction mechanism is of great significance in both nicotine withdrawal and brain science. The detection of addiction-related brain connectivity using functional magnetic resonance imaging (fMRI) is a critical step in study of this mechanism. However, it is challenging to accurately estimate addiction-related brain connectivity due to the low-signal-to-noise ratio of fMRI and the issue of small sample size. In this work, a prior-embedding graph generative adversarial network (PG-GAN) is proposed to capture addiction-related brain connectivity accurately. By designing a dual-generator-based scheme, the addiction-related connectivity generator is employed to learn the feature map of addiction connection, while the reconstruction generator is used for sample reconstruction. Moreover, a bidirectional mapping mechanism is designed to maintain the consistency of sample distribution in the latent space so that addiction-related brain connectivity can be estimated more accurately. The proposed model utilizes prior knowledge embeddings to reduce the search space so that the model can better understand the latent distribution for the issue of small sample size. Experimental results demonstrate the effectiveness of the proposed PG-GAN.

Morphology-based deep learning enables accurate detection of senescence in mesenchymal stem cell cultures.

BACKGROUND: Cell senescence is a sign of aging and plays a significant role in the pathogenesis of age-related disorders. For cell therapy, senescence may compromise the quality and efficacy of cells, posing potential safety risks. Mesenchymal stem cells (MSCs) are currently undergoing extensive research for cell therapy, thus necessitating the development of effective methods to evaluate senescence. Senescent MSCs exhibit distinctive morphology that can be used for detection. However, morphological assessment during MSC production is often subjective and uncertain. New tools are required for the reliable evaluation of senescent single cells on a large scale in live imaging of MSCs. RESULTS: We have developed a successful morphology-based Cascade region-based convolution neural network (Cascade R-CNN) system for detecting senescent MSCs, which can automatically locate single cells of different sizes and shapes in multicellular images and assess their senescence state. Additionally, we tested the applicability of the Cascade R-CNN system for MSC senescence and examined the correlation between morphological changes with other senescence indicators. CONCLUSIONS: This deep learning has been applied for the first time to detect senescent MSCs, showing promising performance in both chronic and acute MSC senescence. The system can be a labor-saving and cost-effective option for screening MSC culture conditions and anti-aging drugs, as well as providing a powerful tool for non-invasive and real-time morphological image analysis integrated into cell production.

Prior-Guided Adversarial Learning With Hypergraph for Predicting Abnormal Connections in Alzheimer's Disease.

Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.

Cell classification with worse-case boosting for intelligent cervical cancer screening.

Cell classification underpins intelligent cervical cancer screening, a cytology examination that effectively decreases both the morbidity and mortality of cervical cancer. This task, however, is rather challenging, mainly due to the difficulty of collecting a training dataset representative sufficiently of the unseen test data, as there are wide variations of cells' appearance and shape at different cancerous statuses. This difficulty makes the classifier, though trained properly, often classify wrongly for cells that are underrepresented by the training dataset, eventually leading to a wrong screening result. To address it, we propose a new learning algorithm, called worse-case boosting, for classifiers effectively learning from under-representative datasets in cervical cell classification. The key idea is to learn more from worse-case data for which the classifier has a larger gradient norm compared to other training data, so these data are more likely to correspond to underrepresented data, by dynamically assigning them more training iterations and larger loss weights for boosting the generalizability of the classifier on underrepresented data. We achieve this idea by sampling worse-case data per the gradient norm information and then enhancing their loss values to update the classifier. We demonstrate the effectiveness of this new learning algorithm on two publicly available cervical cell classification datasets (the two largest ones to the best of our knowledge), and positive results (4% accuracy improvement) yield in the extensive experiments. The source codes are available at: https://github.com/YouyiSong/Worse-Case-Boosting.

Gated CNN-Transformer Network for Automatic Cardiovascular Diagnosis using 12-lead Electrocardiogram.

12-lead electrocardiogram (ECG) is a widely used method in the diagnosis of cardiovascular disease (CVD). With the increase in the number of CVD patients, the study of accurate automatic diagnosis methods via ECG has become a research hotspot. The use of deep learning-based methods can reduce the influence of human subjectivity and improve the diagnosis accuracy. In this paper, we propose a 12-lead ECG automatic diagnosis method based on channel features and temporal features fusion. Specifically, we design a gated CNN-Transformer network, in which the CNN block is used to extract signal embeddings to reduce data complexity. The dual-branch transformer structure is used to effectively extract channel and temporal features in low-dimensional embeddings, respectively. Finally, the features from the two branches are fused by the gating unit to achieve automatic CVD diagnosis from 12-lead ECG. The proposed end-to-end approach has more competitive performance than other deep learning algorithms, which achieves an overall diagnostic accuracy of 85.3% in the 12-lead ECG dataset of CPSC-2018.

USBDAN: Unsupervised Scale-aware and Boundary-aware Domain Adaptive Network for Gastric Tumor Segmentation.

Accurate segmentation of gastric tumors from computed tomography (CT) images provides useful image information for guiding the diagnosis and treatment of gastric cancer. Researchers typically collect datasets from multiple medical centers to increase sample size and representation, but this raises the issue of data heterogeneity. To this end, we propose a new cross-center 3D tumor segmentation method named unsupervised scale-aware and boundary-aware domain adaptive network (USBDAN), which includes a new 3D neural network that efficiently bridges an Anisotropic neural network and a Transformer (AsTr) for extracting multi-scale features from the CT images with anisotropic resolution, and a scale-aware and boundary-aware domain alignment (SaBaDA) module for adaptively aligning multi-scale features between two domains and enhancing tumor boundary drawing based on location-related information drawn from each sample across all domains. We evaluate the proposed method on an in-house CT image dataset collected from four medical centers. Our results demonstrate that the proposed method outperforms several state-of-the-art methods.

Dense Swin Transformer for Classification of Thyroid Nodules.

As an early sign of thyroid cancer, thyroid nodules are the most common nodular lesions. As a non-invasive imaging method, ultrasound is widely used in the diagnosis of benign and malignant thyroid nodules. As there is no obvious difference in appearance between the two types of thyroid nodules, and the contrast with the surrounding muscle tissue is too low, it is difficult to distinguish the benign and malignant nodules. Therefore, a dense nodal Swin-Transformer(DST) method for the diagnosis of thyroid nodules is proposed in this paper. Image segmentation is carried out through patch, and feature maps of different sizes are constructed in four stages, which consider different information of each layer of features. In each stage block, a dense connection mechanism is used to make full use of multi-layer features and effectively improve the diagnostic performance. The experimental results of multi-center ultrasound data collected from 17 hospitals show that the accuracy of the proposed method is 87.27%, the sensitivity is 88.63%, and the specific effect is 85.16%, which verifies that the proposed algorithm has the potential to assist clinical practice.

Fused Brain Functional Connectivity Network and Edge-attention Graph Convolution Network for Fibromyalgia Syndrome Diagnosis.

Fibromyalgia syndrome (FMS) is a type of rheumatology that seriously affects the normal life of patients. Due to the complex clinical manifestations of FMS, it is challenging to detect FMS. Therefore, an automatic FMS diagnosis model is urgently needed to assist physicians. Brain functional connectivity networks (BFCNs) constructed by resting-state functional magnetic resonance imaging (rs-fMRI) to describe brain functions have been widely used to identify individuals with relevant diseases from normal control (NC). Therefore, we propose a novel model based on BFCN and graph convolutional network (GCN) for automatic FMS diagnosis. Firstly, a novel fused BFCN method is proposed by fusing Pearson's correlation (PC) and low-rank (LR) BFCN, which retains information and reduces data redundancy to construct BFCN. Then we combine the feature of BFCN with non-image information of subjects to obtain nodes and adjacency matrices, which builds a graph with edge attention. Finally, the graph is sent to the GCN layer for FMS diagnosis. Our model is evaluated on the in-house FMS dataset to achieve 82.48% accuracy. The experimental results show that our method outperforms the state-of-the-art competing methods.

A Two-stage Diagnostic Framework for Post-ablation Treatment Response Assessment in Patients with Hepatocellular Carcinoma.

Contrast-enhanced ultrasound (CEUS) video plays an important role in post-ablation treatment response assessment in patients with hepatocellular carcinoma (HCC). However, the assessment of treatment response using CEUS video is challenging due to issues such as high inter-frame data repeatability, small ablation area and poor imaging quality of CEUS video. To address these issues, we propose a two-stage diagnostic framework for post-ablation treatment response assessment in patients with HCC using CEUS video. The first stage is a location stage, which is used to locate the ablation area. At this stage, we propose a Yolov5-SFT to improve the location results of the ablation area and a similarity comparison module (SCM) to reduce data repeatability. The second stage is an assessment stage, which is used for the evaluation of postoperative efficacy. At this stage, we design an EfficientNet-SK to improve assessment accuracy. The Experimental results on the self-collected data show that the proposed framework outperforms other selected algorithms, and can effectively assist doctors in the assessment of post-ablation treatment response.

Graph Convolution and Self-attention Enhanced CNN with Domain Adaptation for Multi-site COVID-19 Diagnosis.

In 2019, coronavirus disease (COVID-19) is an acute disease that can rapidly develop into a very serious state. Therefore, it is of great significance to realize automatic COVID-19 diagnosis. However, due to the small difference in the characteristics of computed tomography (CT) between community acquire pneumonia (CP) and COVID-19, the existing model is unsuitable for the three-class classifications of healthy control, CP and COVID-19. The current model rarely optimizes the data from multiple centers. Therefore, we propose a diagnosis model for COVID-19 patients based on graph enhanced 3D convolution neural network (CNN) and cross-center domain feature adaptation. Specifically, we first design a 3D CNN with graph convolution module to enhance the global feature extraction capability of the CNN. Meanwhile, we use the domain adaptive feature alignment method to optimize the feature distance between different centers, which can effectively realize multi-center COVID-19 diagnosis. Our experimental results achieve quite promising COVID-19 diagnosis results, which show that the accuracy in the mixed dataset is 98.05%, and the accuracy in cross-center tasks are 85.29% and 87.53%.

Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network.

Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.

Brain Structure-Function Fusing Representation Learning Using Adversarial Decomposed-VAE for Analyzing MCI.

Integrating the brain structural and functional connectivity features is of great significance in both exploring brain science and analyzing cognitive impairment clinically. However, it remains a challenge to effectively fuse structural and functional features in exploring the complex brain network. In this paper, a novel brain structure-function fusing-representation learning (BSFL) model is proposed to effectively learn fused representation from diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the decomposition-fusion framework is developed to first decompose the feature space into the union of the uniform and unique spaces for each modality, and then adaptively fuse the decomposed features to learn MCI-related representation. Moreover, a knowledge-aware transformer module is designed to automatically capture local and global connectivity features throughout the brain. Also, a uniform-unique contrastive loss is further devised to make the decomposition more effective and enhance the complementarity of structural and functional features. The extensive experiments demonstrate that the proposed model achieves better performance than other competitive methods in predicting and analyzing MCI. More importantly, the proposed model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.

Specificity-aware Federated Learning with Dynamic Feature Fusion Network for Imbalanced Medical Image Classification.

Recently, federated learning has become a powerful technique for medical image classification due to its ability to utilize datasets from multiple clinical clients while satisfying privacy constraints. However, there are still some obstacles in federated learning. Firstly, most existing methods directly average the model parameters collected by medical clients on the server, ignoring the specificities of the local models. Secondly, class imbalance is a common issue in medical datasets. In this paper, to handle these two challenges, we propose a novel specificity-aware federated learning framework that benefits from an Adaptive Aggregation Mechanism (AdapAM) and a Dynamic Feature Fusion Strategy (DFFS). Considering the specificity of each local model, we set the AdapAM on the server. The AdapAM utilizes reinforcement learning to adaptively weight and aggregate the parameters of local models based on their data distribution and performance feedback for obtaining the global model parameters. For the class imbalance in local datasets, we propose the DFFS to dynamically fuse the features of majority classes based on the imbalance ratio in the min-batch and collaborate the rest of features. We conduct extensive experiments on a dermoscopic dataset and a fundus image dataset. Experimental results show that our method can achieve state-of-the-art results in these two real-world medical applications.

End-to-end Prediction of EGFR Mutation Status with Denseformer.

Accurate genotyping of the epidermal growth factor receptor (EGFR) is critical for the treatment planning of lung adenocarcinoma. Currently, clinical identification of EGFR genotyping highly relies on biopsy and sequence testing which is invasive and complicated. Recent advancements in the integration of computed tomography (CT) imagery with deep learning techniques have yielded a non-invasive and straightforward way for identifying EGFR profiles. However, there are still many limitations for further exploration: 1) most of these methods still require physicians to annotate tumor boundaries, which are time-consuming and prone to subjective errors; 2) most of the existing methods are simply borrowed from computer vision field which does not sufficiently exploit the multi-level features for final prediction. To solve these problems, we propose a Denseformer framework to identify EGFR mutation status in a real end-to-end fashion directly from 3D lung CT images. Specifically, we take the 3D whole-lung CT images as the input of the neural network model without manually labeling the lung nodules. This is inspired by the medical report that the mutational status of EGFR is associated not only with the local tumor nodules but also with the microenvironment surrounded by the whole lung. Besides, we design a novel Denseformer network to fully explore the distinctive information across the different level features. The Denseformer is a novel network architecture that combines the advantages of both convolutional neural network (CNN) and Transformer. Denseformer directly learns from the 3D whole-lung CT images, which preserves the spatial location information in the CT images. To further improve the model performance, we designed a combined Transformer module. This module employs the Transformer Encoder to globally integrate the information of different levels and layers and use them as the basis for the final prediction. The proposed model has been tested on a lung adenocarcinoma dataset collected at the Affiliated Hospital of Zunyi Medical University. Extensive experiments demonstrated the proposed method can effectively extract meaningful features from 3D CT images to make accurate predictions. Compared with other state-of-the-art methods, Denseformer achieves the best performance among current methods using deep learning to predict EGFR mutation status based on a single modality of CT images.

Domain-Generalized EEG Classification With Category-Oriented Feature Decorrelation and Cross-View Consistency Learning.

Generalizing the electroencephalogram (EEG) decoding methods to unseen subjects is an important research direction for realizing practical application of brain-computer interfaces (BCIs). Since distribution shifts across subjects, the performance of most current deep neural networks for decoding EEG signals degrades when dealing with unseen subjects. Domain generalization (DG) aims to tackle this issue by learning invariant representations across subjects. To this end, we propose a novel domain-generalized EEG classification framework, named FDCL, to generalize EEG decoding through category-relevant and -irrelevant Feature Decorrelation and Cross-view invariant feature Learning. Specifically, we first devise data augmented regularization through mixing the segments of same-category features from multiple subjects, which increases the diversity of EEG data by spanning the space of subjects. Furthermore, we introduce feature decorrelation regularization to learn the weights of the augmented EEG trials to remove the dependencies between their features, so that the true mapping relationship between relevant features and corresponding labels can be better established. To further distill subject-invariant EEG feature representations, cross-view consistency learning regularization is introduced to encourage consistent predictions of category-relevant features induced from different augmented EEG views. We seamlessly integrate three complementary regularizations into a unified DG framework to jointly improve the generalizability and robustness of the model on unseen subjects. Experimental results on motor imagery (MI) based EEG datasets validate that the proposed FDCL outperforms the available state-of-the-art methods.

Federated Domain Adaptation via Transformer for Multi-Site Alzheimer's Disease Diagnosis.

In multi-site studies of Alzheimer's disease (AD), the difference of data in multi-site datasets leads to the degraded performance of models in the target sites. The traditional domain adaptation method requires sharing data from both source and target domains, which will lead to data privacy issue. To solve it, federated learning is adopted as it can allow models to be trained with multi-site data in a privacy-protected manner. In this paper, we propose a multi-site federated domain adaptation framework via Transformer (FedDAvT), which not only protects data privacy, but also eliminates data heterogeneity. The Transformer network is used as the backbone network to extract the correlation between the multi-template region of interest features, which can capture the brain abundant information. The self-attention maps in the source and target domains are aligned by applying mean squared error for subdomain adaptation. Finally, we evaluate our method on the multi-site databases based on three AD datasets. The experimental results show that the proposed FedDAvT is quite effective, achieving accuracy rates of 88.75%, 69.51%, and 69.88% on the AD vs. NC, MCI vs. NC, and AD vs. MCI two-way classification tasks, respectively.

MHW-GAN: Multidiscriminator Hierarchical Wavelet Generative Adversarial Network for Multimodal Image Fusion.

Image fusion technology aims to obtain a comprehensive image containing a specific target or detailed information by fusing data of different modalities. However, many deep learning-based algorithms consider edge texture information through loss functions instead of specifically constructing network modules. The influence of the middle layer features is ignored, which leads to the loss of detailed information between layers. In this article, we propose a multidiscriminator hierarchical wavelet generative adversarial network (MHW-GAN) for multimodal image fusion. First, we construct a hierarchical wavelet fusion (HWF) module as the generator of MHW-GAN to fuse feature information at different levels and scales, which avoids information loss in the middle layers of different modalities. Second, we design an edge perception module (EPM) to integrate edge information from different modalities to avoid the loss of edge information. Third, we leverage the adversarial learning relationship between the generator and three discriminators for constraining the generation of fusion images. The generator aims to generate a fusion image to fool the three discriminators, while the three discriminators aim to distinguish the fusion image and edge fusion image from two source images and the joint edge image, respectively. The final fusion image contains both intensity information and structure information via adversarial learning. Experiments on public and self-collected four types of multimodal image datasets show that the proposed algorithm is superior to the previous algorithms in terms of both subjective and objective evaluation.